Backport muon shower (un)packing and DQM to 12_2_X #37225

dinyar · 2022-03-13T18:46:40Z

PR description:

This is a backport PR comprising multiple original ones. It contains unpackers and packers for hadronic showers at the uGMT inputs and outputs as well as the DQM for muon showers.

Ideally this should be used in the P5 DQM as soon as possible, could you let me know if this isn't the correct branch to target for this?

attn: @vukasinmilosevic

PR validation:

Ran runTheMatrix.py -l limited -i all --ibeos as well as scram b runtests after rebasing/cherry-picking.

if this PR is a backport please specify the original PR and why you need to backport that PR:

Original PRs:

I would like to have these changes available in the DQM at P5. They are not required for T0 or HLT.

…s for RegionalMuonCand

We were generating this error by looking at the wrong bits.

Two out of six words from the payload are used for showers, we were unfortunately off by one.

Also make error message if we're in danger of overwriting data with muon shower info more useful.

This didn't actually break anything, but caused confusing errors when trying to pack (always empty) showers for OMTF.

cmsbuild · 2022-03-13T18:47:07Z

A new Pull Request was created by @dinyar (Dinyar Rabady) for CMSSW_12_2_X.

It involves the following packages:

DQM/L1TMonitor (dqm)
DataFormats/L1TMuon (l1)
DataFormats/L1Trigger (l1)
EventFilter/L1TRawToDigi (l1)
L1Trigger/L1TMuon (l1)
L1Trigger/L1TMuonEndCap (l1)

@epalencia, @emanueleusai, @ahmad3213, @cmsbuild, @rekovic, @jfernan2, @pmandrik, @cecilecaillol, @pbo0, @rvenditti can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @rovere, @kreczko, @thomreis this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

jfernan2 · 2022-03-13T18:49:19Z

12_2_X at present is the right branch to target Online DQM, it will be tested tomorrow

jfernan2 · 2022-03-13T18:50:18Z

test parameters:

addpkg = DQM/Integration

jfernan2 · 2022-03-13T18:51:14Z

please test

cmsbuild · 2022-03-13T22:22:26Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-73f863/23074/summary.html
COMMIT: 26727cc
CMSSW: CMSSW_12_2_X_2022-03-13-0000/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37225/23074/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

The workflows 1001.0, 1000.0, 136.88811, 136.874, 136.8311, 136.793, 136.7611, 136.731, 4.22 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 42
DQMHistoTests: Total histograms compared: 3251294
DQMHistoTests: Total failures: 72
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3251200
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 8.73 KiB( 41 files compared)
DQMHistoSizes: changed ( 11634.0,... ): 1.261 KiB L1T/L1TStage2uGMT
DQMHistoSizes: changed ( 11634.0,... ): 0.194 KiB L1TEMU/L1TdeStage2uGMT
Checked 177 log files, 37 edm output root files, 42 DQM output files
TriggerResults: no differences found

perrotta · 2022-03-14T06:36:22Z

@dinyar could you please verify and confirm that whatever is backported here for 12_2_X is already available in the master AND in 12_3_X?
12_3_X will become rather soon the new datataking releas in place of 12_2_X, and whatever is implemented in 12_2 cannot miss in the newer cycle that will subsitute it at P5

dinyar · 2022-03-14T10:13:08Z

Hi @perrotta,

Yes, I confirm that is the case. (I checked that the PRs listed all had milestone CMSSW_12_3_X and that all commits in this PR are present in those PRs.)

Cheers,
Dinyar

jfernan2 · 2022-03-14T11:56:20Z

+1
Tested successfully in Online DQM at P5 (no crashes). Minor differences due to changes in the PR description:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_12_2_X_2022-03-13-0000+73f863/48913/dqm-histo-comparison-summary.html

perrotta · 2022-03-14T14:58:57Z

@cms-sw/l1-l2 we are going to buind another 12_2 release in short: if you are happy with this multiple backport PR, which faithfully reproduces the four PRs listed in the PR description and already merged in 12_3_X and in the master, please sign it, so that it can also be included.

cecilecaillol · 2022-03-14T15:01:23Z

+l1

cmsbuild · 2022-03-14T15:01:48Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_4_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-03-14T15:04:31Z

+1

It faithfully reproduces the four PRs listed in the PR description and already merged in 12_3_X and in the master

dinyar · 2022-03-21T08:39:08Z

+1 Tested successfully in Online DQM at P5 (no crashes). Minor differences due to changes in the PR description: https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_12_2_X_2022-03-13-0000+73f863/48913/dqm-histo-comparison-summary.html

Hi @jfernan2,

I think this hasn't been deployed to online DQM yet, correct? It's not urgent, but do you have a timeline already for when I should expect it? (If there is something needed from my side please also let me know.)

Cheers,
Dinyar

ahmad3213 · 2022-03-21T08:57:28Z

I think this hasn't been deployed to online DQM yet, correct?

Correct

It's not urgent, but do you have a timeline already for when I should expect it? (If there is something needed from my side please also let me know.)

We can try to put in production today

dinyar · 2022-03-21T15:00:31Z

It's not urgent, but do you have a timeline already for when I should expect it? (If there is something needed from my side please also let me know.)

We can try to put in production today

Ok, thanks! If not today anytime this week would be great.

anigamova · 2022-03-21T16:43:54Z

Hi @dinyar,
This PR is deployed in prduction since the run 349126

dinyar and others added 18 commits March 8, 2022 17:41

Use the same interface to set TF identifiers for RegionalMuonShower a…

0e7d663

…s for RegionalMuonCand

Add (un)packing functions for RegionalMuonShower

892b3db

Add tokens and collections for unpacked muon showers

2f3ae1c

(Un)pack regional muon showers

200ae97

(Un)packing of hadronic showers at the uGMT output

37281c1

Add muon shower emulators to Phase2 sequence

925ab4f

Explicitly name constant indicating number of output copies

1133d2e

Update shower (un)packing function with new specification

d9783db

Fix error inhibiting muon shower packing

41b46f6

We were generating this error by looking at the wrong bits.

Select correct word in payload for muon showers

9a78fe0

Two out of six words from the payload are used for showers, we were unfortunately off by one.

Don't write Run-2 EMTF debug data for Run-3

a8c6d4f

Also make error message if we're in danger of overwriting data with muon shower info more useful.

Only pack muon showers if they're valid

dafa165

This didn't actually break anything, but caused confusing errors when trying to pack (always empty) showers for OMTF.

Fill old fields with something to avoid L1TNtuple crashes

f046d30

Make MuonShower data format more descriptive

69399a9

Enable unpacking of uGMT output muon showers

841964f

Enable shower unpacking for the uGT inputs

bca02af

Only fill RegionalMuonShower if it's valid

a30d4d8

Add muon shower plots to uGMT online DQM

26727cc

cmsbuild added this to the CMSSW_12_2_X milestone Mar 13, 2022

cmsbuild added dqm-pending l1-pending orp-pending pending-signatures tests-pending labels Mar 13, 2022

cmsbuild added tests-started and removed tests-pending labels Mar 13, 2022

cmsbuild added tests-approved and removed tests-started labels Mar 13, 2022

cmsbuild added dqm-approved and removed dqm-pending labels Mar 14, 2022

cmsbuild added fully-signed l1-approved and removed l1-pending pending-signatures labels Mar 14, 2022

cmsbuild added orp-approved and removed orp-pending labels Mar 14, 2022

cmsbuild merged commit 7ffc6f4 into cms-sw:CMSSW_12_2_X Mar 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backport muon shower (un)packing and DQM to 12_2_X #37225

Backport muon shower (un)packing and DQM to 12_2_X #37225

dinyar commented Mar 13, 2022

cmsbuild commented Mar 13, 2022

jfernan2 commented Mar 13, 2022

jfernan2 commented Mar 13, 2022

jfernan2 commented Mar 13, 2022

cmsbuild commented Mar 13, 2022

perrotta commented Mar 14, 2022

dinyar commented Mar 14, 2022

jfernan2 commented Mar 14, 2022

perrotta commented Mar 14, 2022

cecilecaillol commented Mar 14, 2022

cmsbuild commented Mar 14, 2022

perrotta commented Mar 14, 2022

dinyar commented Mar 21, 2022

ahmad3213 commented Mar 21, 2022

dinyar commented Mar 21, 2022

anigamova commented Mar 21, 2022

Backport muon shower (un)packing and DQM to 12_2_X #37225

Backport muon shower (un)packing and DQM to 12_2_X #37225

Conversation

dinyar commented Mar 13, 2022

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Mar 13, 2022

jfernan2 commented Mar 13, 2022

jfernan2 commented Mar 13, 2022

jfernan2 commented Mar 13, 2022

cmsbuild commented Mar 13, 2022

Comparison Summary

perrotta commented Mar 14, 2022

dinyar commented Mar 14, 2022

jfernan2 commented Mar 14, 2022

perrotta commented Mar 14, 2022

cecilecaillol commented Mar 14, 2022

cmsbuild commented Mar 14, 2022

perrotta commented Mar 14, 2022

dinyar commented Mar 21, 2022

ahmad3213 commented Mar 21, 2022

dinyar commented Mar 21, 2022

anigamova commented Mar 21, 2022